Hidden Markov Models and Dynamic Programming

1 Last week: stochastic part-of-speech tagging Last week we reviewed parts-of-speech, which are linguistic categories of words. These categories are defined in terms of syntactic or morphological behaviour. Parts-of-speech for English traditionally include: Nouns are concrete or abstract entity; Pronouns substitute for nouns; Adjective modify nouns; Verbs are actions or states of being; Adverbs modify adjectives, verbs or other adverbs; Prepositions express relations; and Conjuctions connect expressions. Linguists typically distinguish between many more types however. For example, the tagset you will use in the assignment exercises is The Penn Treebank Tagset, which contains 45 parts-of-speech. Automatic part-of-speech tagging can be achieved using a large set of heuristics, but defining such heuristics is a very time-consuming task. Instead we can use a stochastic method, which, given a sequence of observed words w 1 we want to determine the most probable corresponding sequence of classes t̂1 . t̂1 = arg max t1 P (t1 |w 1 )